Regular Birth Times for Markov Processes

Markov Decision Processes with Constrained Stopping Times

The optimization problem for a stopped Markov decision process is considered to be taken over stopping times constrained so that E 5 for some xed > 0. We introduce the concept of a randomized stationary stopping time which is a mixed extension of the entry time of a stopping region and prove the existence of an optimal constrained pair of stationary policy and stopping time by utilizing a Lagra...

Extinction times in Multitype Markov Branching Processes

In this paper, a distributional approximation to the time to extinction in a subcritical continuous-time Markov branching process is derived. A limit theorem for this distribution is established and the error in the approximation is quantified. The accuracy of the approximation is illustrated in an epidemiological example. Since Markov branching processes serve as approximations to nonlinear ep...

An Axiomatization for Regular Processes in Times Branching Bisimulation

Klusener introduced a timed variant of branching bisimulation. In this paper it is shown that Klusener's axioms for nite process terms, together with two standard axioms for recursion, make a complete axiomatization for regular processes modulo timed branching bisimulation.

Spatial Birth-and-Death Markov Processes

Relative entropy and waiting times for continuous-time Markov processes

For discrete-time stochastic processes, there is a close connection between return/waiting times and entropy. Such a connection cannot be straightforwardly extended to the continuous-time setting. Contrarily to the discrete-time case one does need a reference measure and so the natural object is relative entropy rather than entropy. In this paper we elaborate on this in the case of continuous-t...